A Comparative Study on Representation of Web Pages in Automatic Text Categorization

With many web sites appearing everyday, it has become increasingly difficult to keep the web directories up-to-date and growing. The interest in the usage of machine learning on automatic text categorization is further stimulated with this intensive growth of World Wide Web. We believe that Web page classification is significantly different from a traditional text classification because of the presence of some additional information, provided by the HTML structure and by the presence of the hyperlinks. In this paper, our objective is to analyze different combinations to represent the training documents and the test documents for SVM classifier. Our experiments show that in addition to the content of the web site, using further the META data and the extended inbound anchor text information in representing the Web sites, enhances the performance of the classification. Moreover, utilizing the expected entropy loss values for the purpose of weighting the term frequencies in the feature vector provides further performance enhancement in SVM classifier.